Theory of Mind in Intelligent User Interfaces

The ability to read the mind and the language of the eyes are two cornerstones for the development of social function and emotional intelligence in humans [1]. Mind reading is the ability to infer other people’s mental state and use that to make sense of and predict their behavior. A lack of or impairment in the theory of mind (mindblindness) is thought to be the primary inhibitor of emotion understanding and social intelligence in individuals with autism [2]. The language of the eyes communicates a person’s basic emotion as well as complex mental states [4]. In this paper we show how reading the mind and the eyes can be used to build the framework for socially intelligent interfaces that recognize and predict their user’s emotion state. The paper presents a mind-reading architecture based on an Eye Language Interpreter and Baron-Cohen’s Mindreading System. This architecture enables the recognition and prediction of complex mental states, allowing for more natural man-machine interaction. The paper also identifies automated facial feature analysis as a key enabling technology required in order to fully implement the architecture, and describes ongoing work on the implementation of specific work in that area.